Exploring Hidden Semantics in Neural Networks with Symbolic Regression

Many recent studies focus on developing mechanisms to explain the black-box behaviors of neural networks (NNs). However, little work has been done to extract the potential hidden semantics (mathematical representation) of a neural network. A succinct and explicit mathematical representation of a NN model could improve the understanding and interpretation of its behaviors. To address this need, we propose a novel symbolic regression method for neural works (called SRNet) to discover the mathematical expressions of a NN. SRNet creates a Cartesian genetic programming (NNCGP) to represent the hidden semantics of a single layer in a NN. It then leverages a multi-chromosome NNCGP to represent hidden semantics of all layers of the NN. The method uses a (1+) evolutionary strategy (called MNNCGP-ES) to extract the final mathematical expressions of all layers in the NN. Experiments on 12 symbolic regression benchmarks and 5 classification benchmarks show that SRNet not only can reveal the complex relationships between each layer of a NN but also can extract the mathematical representation of the whole NN. Compared with LIME and MAPLE, SRNet has higher interpolation accuracy and trends to approximate the real model on the practical dataset

Fashionable Technology, Fashion Waves, and Post-Adoption Regret and Satisfaction

This research attempts to understand user adoption of fashionable technologies (e.g., iPhone or iPad) and the influence of fashion waves on adopters of both fashionable and non-fashionable technologies. A research model was developed based on the regret theory. We tested the model by examining 20,122 customer reviews collected from Amazon.com. A theory-driven naïve Bayes classifier was developed to analyze the regret elements of customer reviews automatically. The data largely supported the research model. Specifically, we found that adopters of non-fashionable phones experience higher levels of regret and lower satisfaction during the fashion wave, i.e., when a new fashionable phone was released. In contrast, adopters of earlier editions of fashionable phones welcomed the new fashionable phone, displaying lower levels of regret and higher satisfaction during the fashion wave period. The findings have significant implications for information systems research and practices

Identifying Trippers and Non-Trippers Based on Knee Kinematics During Obstacle-Free Walking

Trips are a major cause of falls. Sagittal-plane kinematics affect clearance between the foot and obstacles, however, it is unclear which kinematic measures during obstacle-free walking are associated with avoiding a trip when encountering an obstacle. The purpose of this study was to determine kinematic factors during obstacle-free walking that are related to obstacle avoidance ability. It was expected that successful obstacle avoidance would be associated with greater peak flexion/dorsiflexion and range of motion (ROM), and differences in timing of peak flexion/dorsiflexion during swing of obstacle-free walking for the hip, knee and ankle. Three-dimensional kinematics were recorded as 35 participants (young adults age 18–45 (N = 10), older adults age 65+ without a history of falls (N = 10), older adults age 65+ who had fallen in the last six months (N = 10), and individuals who had experienced a stroke more than six months earlier (N = 5)) walked on a treadmill, under obstacle-free walking conditions with kinematic features calculated for each stride. A separate obstacle avoidance task identified trippers (multiple obstacle contact) and non-trippers. Linear discriminant analysis with sequential feature selection classified trippers and non-trippers based on kinematics during obstacle-free walking. Differences in classification performance and selected features (knee ROM and timing of peak knee flexion during swing) were evaluated between trippers and non-trippers. Non-trippers had greater knee ROM (P = .001). There was no significant difference in classification performance (P = .193). Individuals with reduced knee ROM during obstacle-free walking may have greater difficulty avoiding obstacles

Using data-driven sublanguage pattern mining to induce knowledge models: application in medical image reports knowledge representation

Background: The use of knowledge models facilitates information retrieval, knowledge base development, and therefore supports new knowledge discovery that ultimately enables decision support applications. Most existing works have employed machine learning techniques to construct a knowledge base. However, they often suffer from low precision in extracting entity and relationships. In this paper, we described a data-driven sublanguage pattern mining method that can be used to create a knowledge model. We combined natural language processing (NLP) and semantic network analysis in our model generation pipeline. Methods: As a use case of our pipeline, we utilized data from an open source imaging case repository, Radiopaedia.org, to generate a knowledge model that represents the contents of medical imaging reports. We extracted entities and relationships using the Stanford part-of-speech parser and the “Subject:Relationship:Object” syntactic data schema. The identified noun phrases were tagged with the Unified Medical Language System (UMLS) semantic types. An evaluation was done on a dataset comprised of 83 image notes from four data sources. Results: A semantic type network was built based on the co-occurrence of 135 UMLS semantic types in 23,410 medical image reports. By regrouping the semantic types and generalizing the semantic network, we created a knowledge model that contains 14 semantic categories. Our knowledge model was able to cover 98% of the content in the evaluation corpus and revealed 97% of the relationships. Machine annotation achieved a precision of 87%, recall of 79%, and F-score of 82%. Conclusion: The results indicated that our pipeline was able to produce a comprehensive content-based knowledge model that could represent context from various sources in the same domain

Taylor Genetic Programming for Symbolic Regression

Genetic programming (GP) is a commonly used approach to solve symbolic regression (SR) problems. Compared with the machine learning or deep learning methods that depend on the pre-defined model and the training dataset for solving SR problems, GP is more focused on finding the solution in a search space. Although GP has good performance on large-scale benchmarks, it randomly transforms individuals to search results without taking advantage of the characteristics of the dataset. So, the search process of GP is usually slow, and the final results could be unstable. To guide GP by these characteristics, we propose a new method for SR, called Taylor genetic programming (TaylorGP). TaylorGP leverages a Taylor polynomial to approximate the symbolic equation that fits the dataset. It also utilizes the Taylor polynomial to extract the features of the symbolic equation: low order polynomial discrimination, variable separability, boundary, monotonic, and parity. GP is enhanced by these Taylor polynomial techniques. Experiments are conducted on three kinds of benchmarks: classical SR, machine learning, and physics. The experimental results show that TaylorGP not only has higher accuracy than the nine baseline methods, but also is faster in finding stable results

Understanding patient needs and gaps in radiology reports through online discussion forum analysis

Our objective is to investigate patient needs and understand information gaps in radiology reports using patient questions that were posted on online discussion forums. We leveraged online question and answer platforms to collect questions posted by patients to understand current gaps and patient needs. We retrieved six hundred fifty-nine (659) questions using the following sites: Yahoo Answers, Reddit.com, Quora, and Wiki Answers. The questions retrieved were analyzed and the major themes and topics were identified. The questions retrieved were classified into eight major themes. The themes were related to the following topics: radiology report, safety, price, preparation, procedure, meaning, medical staff, and patient portal. Among the 659 questions, 35.50% were concerned with the radiology report. The most common question topics in the radiology report focused on patient understanding of the radiology report (62 of 234 [26.49%]), image visualization (53 of 234 [22.64%]), and report representation (46 of 234 [19.65%]). We also found that most patients were concerned about understanding the MRI report (32%; n = 143) compared with the other imaging modalities (n = 434). Using online discussion forums, we discussed major unmet patient needs and information gaps in radiology reports. These issues could be improved to enhance radiology design in the future

A Deep Learning Study on Osteosarcoma Detection from Histological Images

In the U.S, 5-10\% of new pediatric cases of cancer are primary bone tumors. The most common type of primary malignant bone tumor is osteosarcoma. The intention of the present work is to improve the detection and diagnosis of osteosarcoma using computer-aided detection (CAD) and diagnosis (CADx). Such tools as convolutional neural networks (CNNs) can significantly decrease the surgeon's workload and make a better prognosis of patient conditions. CNNs need to be trained on a large amount of data in order to achieve a more trustworthy performance. In this study, transfer learning techniques, pre-trained CNNs, are adapted to a public dataset on osteosarcoma histological images to detect necrotic images from non-necrotic and healthy tissues. First, the dataset was preprocessed, and different classifications are applied. Then, Transfer learning models including VGG19 and Inception V3 are used and trained on Whole Slide Images (WSI) with no patches, to improve the accuracy of the outputs. Finally, the models are applied to different classification problems, including binary and multi-class classifiers. Experimental results show that the accuracy of the VGG19 has the highest, 96\%, performance amongst all binary classes and multiclass classification. Our fine-tuned model demonstrates state-of-the-art performance on detecting malignancy of Osteosarcoma based on histologic images

SmartTrack: Efficient Predictive Race Detection

Widely used data race detectors, including the state-of-the-art FastTrack algorithm, incur performance costs that are acceptable for regular in-house testing, but miss races detectable from the analyzed execution. Predictive analyses detect more data races in an analyzed execution than FastTrack detects, but at significantly higher performance cost. This paper presents SmartTrack, an algorithm that optimizes predictive race detection analyses, including two analyses from prior work and a new analysis introduced in this paper. SmartTrack's algorithm incorporates two main optimizations: (1) epoch and ownership optimizations from prior work, applied to predictive analysis for the first time; and (2) novel conflicting critical section optimizations introduced by this paper. Our evaluation shows that SmartTrack achieves performance competitive with FastTrack-a qualitative improvement in the state of the art for data race detection.Comment: Extended arXiv version of PLDI 2020 paper (adds Appendices A-E) #228 SmartTrack: Efficient Predictive Race Detectio

Proposed Questions to Assess the Extent of Knowledge in Understanding the Radiology Report Language

Radiotherapy and diagnostic imaging play a significant role in medical care. The amount of patient participation and communication can be increased by helping patients understand radiology reports. There is insufficient information on how to measure a patient’s knowledge of a written radiology report. The goal of this study is to design a tool that will measure patient literacy of radiology reports. A radiological literacy tool was created and evaluated as part of the project. There were two groups of patients: control and intervention. A sample radiological report was provided to each group for reading. After reading the report, the groups were quizzed to see how well they understood the report. The participants answered the questions and the correlation between the understanding of the radiology report and the radiology report literacy questions was calculated. The correlations between radiology report literacy questions and radiology report understanding for the intervention and control groups were 0.522, p \u3c 0.001, and 0.536, p \u3c 0.001, respectively. Our radiology literacy tool demonstrated a good ability to measure the awareness of radiology report understanding (area under the receiver operator curve in control group (95% CI: 0.77 (0.71–0.81)) and intervention group (95% CI: 0.79 (0.74–0.84))). We successfully designed a tool that can measure the radiology literacy of patients. This tool is one of the first to measure the level of patient knowledge in the field of radiology understanding

Deactivation of Akt by a Small Molecule Inhibitor Targeting Pleckstrin Homology Domain and Facilitating Akt Ubiquitination

The phosphatidylinositol-3,4,5-triphosphate (PIP3) binding function of pleckstrin homology (PH) domain is essential for the activation of oncogenic Akt/PKB kinase. Following the PIP3-mediated activation at the membrane, the activated Akt is subjected to other regulatory events, including ubiquitination-mediated deactivation. Here, by identifying and characterizing an allosteric inhibitor, SC66, we show that the facilitated ubiquitination effectively terminates Akt signaling. Mechanistically, SC66 manifests a dual inhibitory activity that directly interferes with the PH domain binding to PIP3 and facilitates Akt ubiquitination. A known PH domain-dependent allosteric inhibitor, which stabilizes Akt, prevents the SC66-induced Akt ubiquitination. A cancer-relevant Akt1 (e17k) mutant is unstable, making it intrinsically sensitive to functional inhibition by SC66 in cellular contexts in which the PI3K inhibition has little inhibitory effect. As a result of its dual inhibitory activity, SC66 manifests a more effective growth suppression of transformed cells that contain a high level of Akt signaling, compared with other inhibitors of PIP3/Akt pathway. Finally, we show the anticancer activity of SC66 by using a soft agar assay as well as a mouse xenograft tumor model. In conclusion, in this study, we not only identify a dual-function Akt inhibitor, but also demonstrate that Akt ubiquitination could be chemically exploited to effectively facilitate its deactivation, thus identifying an avenue for pharmacological intervention in Akt signaling